Agile software development

Software development process
Activities and steps
Requirements · Specification
Architecture · Design
Implementation · Testing
Deployment · Maintenance
Methodologies
Agile · Cleanroom · Iterative
RAD · RUP · Spiral
Waterfall · XP · Lean
Scrum · V-Model · TDD
Supporting disciplines
Configuration management
Documentation
Quality assurance (SQA)
Project management
User experience design
Tools
Compiler · Debugger · Profiler
GUI designer · IDE

Agile software development is a group of software development methodologies based on iterative and incremental development, where requirements and solutions evolve through collaboration between self-organizing, cross-functional teams. The Agile Manifesto[1] introduced the term in 2001.

Proponents of agile methods believe that they promote a disciplined project management process for software development that encourages frequent inspection and adaptation, teamwork, self-organization and accountability, a set of engineering best practices for rapid delivery of high-quality software, and a business approach that aligns development with customer needs and company goals.

Detractors of "agile methods" contend that this purported "methodology" is little more than an excuse to avoid planning, analysis and structure methods, and that it is little more than a "quick and dirty" approach to software development. It is driven by software developers who don't want the intrusive influence upon their "agility" that comes with proper documentation understood by the users and managers; is an engineering "worst practice;" is a candidate only for processes that have not been previously automated, or where there is a low bar for minimal functional requirements; and is a significant contributing factor to why many software development projects fail.

Contents

History

Predecessors

Jeff Sutherland, one of the developers of the Scrum agile software development process

Incremental software development methods have been traced back to 1957.[2] In 1974, a paper by E. A. Edmonds introduced an adaptive software development process.[3]

So-called "lightweight" software development methods evolved in the mid-1990s as a reaction against so-called "heavyweight" methods, which were characterized by their critics as a heavily regulated, regimented, micromanaged, waterfall model of development. Proponents of lightweight methods, and now agile methods, contend that they are a return to development practices from early in the history of software development.[2]

Early implementation of lightweight methods include Scrum (1995), Crystal Clear, Extreme Programming (1996), Adaptive Software Development, Feature Driven Development, and Dynamic Systems Development Method (DSDM) (1995). These are now typically referred to as agile methodologies, after the Agile Manifesto published in 2001.[4]

Agile Manifesto

In February 2001, 17 software developers[5] met at a ski resort in Snowbird, Utah, to discuss lightweight development methods. They published the Manifesto for Agile Software Development[1] to define the approach now known as agile software development. Some of the manifesto's authors formed the Agile Alliance, a non-profit organization that promotes software development according to the manifesto's principles.

Agile Manifesto reads, in its entirety, as follows:[1]

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.

Twelve principles underlie the Agile Manifesto, including:[6]

In 2005, a group headed by Alistair Cockburn and Jim Highsmith wrote an addendum of project management principles, the Declaration of Interdependence,[7] to guide software project management according to agile development methods.

Common characteristics

Pair programming, an agile development technique

There are many specific agile development methods. Most promote development, teamwork, collaboration, and process adaptability throughout the life-cycle of the project.

Agile methods break tasks into small increments with minimal planning, and do not directly involve long-term planning. Iterations are short time frames (timeboxes) that typically last from one to four weeks. Each iteration involves a team working through a full software development cycle including planning, requirements analysis, design, coding, unit testing, and acceptance testing when a working product is demonstrated to stakeholders. This helps minimize overall risk, and lets the project adapt to changes quickly. Stakeholders produce documentation as required. An iteration may not add enough functionality to warrant a market release, but the goal is to have an available release (with minimal bugs) at the end of each iteration.[8] Multiple iterations may be required to release a product or new features.

Team composition in an agile project is usually cross-functional and self-organizing without consideration for any existing corporate hierarchy or the corporate roles of team members. Team members normally take responsibility for tasks that deliver the functionality an iteration requires. They decide individually how to meet an iteration's requirements.

Agile methods emphasize face-to-face communication over written documents when the team is all in the same location. When a team works in different locations, they maintain daily contact through videoconferencing, voice, e-mail, etc.

Most agile teams work in a single open office (called a bullpen), which facilitates such communication. Team size is typically small (5-9 people) to help make team communication and team collaboration easier. Larger development efforts may be delivered by multiple teams working toward a common goal or different parts of an effort. This may also require a coordination of priorities across teams.

No matter what development disciplines are required, each agile team will contain a customer representative. This person is appointed by stakeholders to act on their behalf and makes a personal commitment to being available for developers to answer mid-iteration problem-domain questions. At the end of each iteration, stakeholders and the customer representative review progress and re-evaluate priorities with a view to optimizing the return on investment and ensuring alignment with customer needs and company goals.

Most agile implementations use a routine and formal daily face-to-face communication among team members. This specifically includes the customer representative and any interested stakeholders as observers. In a brief session, team members report to each other what they did the previous day, what they intend to do today, and what their roadblocks are. This standing face-to-face communication prevents problems from being hidden.

Agile emphasizes working software as the primary measure of progress. This, combined with the preference for face-to-face communication, produces less written documentation than other methods. The agile method encourages stakeholders to prioritize wants with other iteration outcomes based exclusively on business value perceived at the beginning of the iteration.

Specific tools and techniques such as continuous integration, automated or xUnit test, pair programming, test driven development, design patterns, domain-driven design, code refactoring and other techniques are often used to improve quality and enhance project agility.

Comparison with other methods

Agile methods are sometimes characterized as being at the opposite end of the spectrum from "plan-driven" or "disciplined" methods. This distinction is misleading, as it implies that agile methods are "unplanned" or "undisciplined". Further, agile teams may employ very highly disciplined formal methods. A more accurate distinction is that methods exist on a continuum from "adaptive" to "predictive".[9] Agile methods lie on the "adaptive" side of this continuum.

Adaptive methods focus on adapting quickly to changing realities. When the needs of a project change, an adaptive team changes as well. An adaptive team will have difficulty describing exactly what will happen in the future. The further away a date is, the more vague an adaptive method will be about what will happen on that date. An adaptive team can report exactly what tasks are being done next week, but only which features are planned for next month. When asked about a release six months from now, an adaptive team may only be able to report the mission statement for the release, or a statement of expected value vs. cost.

Predictive methods, in contrast, focus on planning the future in detail. A predictive team can report exactly what features and tasks are planned for the entire length of the development process. Predictive teams have difficulty changing direction. The plan is typically optimized for the original destination and changing direction can cause completed work to be thrown away and done over differently. Predictive teams will often institute a change control board to ensure that only the most valuable changes are considered.

Formal methods, in contrast to adaptive and predictive methods focus on computer science theory with a wide array of types of provers. A formal method attempts to prove the absence of errors with some level of determinism. Some Formal methods are based on model checking and provide counter examples for code that cannot be proven. Generally, mathematical models (often supported through special languages see SPIN model checker) map to assertions about requirements. Formal method are heavily dependent on a tool driven approach, and may be combined with other development approaches. Some provers do not easily scale. Like agile methods, manifestos relevant to high integrity software have been proposed in Crosstalk.

Agile methods have much in common with the "Rapid Application Development" techniques from the 1980/90s as espoused by James Martin and others.

Other iterative development methods

Most agile methods share other iterative and incremental development methods' emphasis on building releasable software in short time periods. Agile development differs from other development models: in this model, time periods are measured in weeks rather than months and work is performed in a highly collaborative manner. Most agile methods also differ by treating their time period as a timebox.

Cowboy coding

Cowboy coding is a derogatory term for software development without a defined or structured method: team members do whatever they feel is right. The Agile approach is sometimes confused with cowboy coding due to its frequent re-evaluation of plans, emphasis on face-to-face communication, and relatively sparse use of documentation. However, Agile teams follow clearly defined, even rigid processes and controls (e.g., deadlines for completion of coding/testing); it is likely the flexibility and adaptability of the overall methodology which causes the confusion. Further, Agile controls offer stronger levels of accountability. The degradation of such controls or procedures can lead to activities that are often categorized as cowboy coding.

Agile methods

Well-known agile software development methods include:

Method tailoring

In the literature, different terms refer to the notion of method adaptation, including ‘method tailoring’, ‘method fragment adaptation’ and ‘situational method engineering’. Method tailoring is defined as:

A process or capability in which human agents through responsive changes in, and dynamic interplays between contexts, intentions, and method fragments determine a system development approach for a specific project situation.[10]

Potentially, almost all agile methods are suitable for method tailoring. Even the DSDM method is being used for this purpose and has been successfully tailored in a CMM context.[11] Situation-appropriateness can be considered as a distinguishing characteristic between agile methods and traditional software development methods, with the latter being relatively much more rigid and prescriptive. The practical implication is that agile methods allow project teams to adapt working practices according to the needs of individual projects. Practices are concrete activities and products that are part of a method framework. At a more extreme level, the philosophy behind the method, consisting of a number of principles, could be adapted (Aydin, 2004).[10]

Extreme Programming (XP) makes the need for method adaptation explicit. One of the fundamental ideas of XP is that no one process fits every project, but rather that practices should be tailored to the needs of individual projects. Partial adoption of XP practices, as suggested by Beck, has been reported on several occasions.[12] A tailoring practice is proposed by Mehdi Mirakhorli which provides sufficient roadmap and guideline for adapting all the practices. RDP Practice is designed for customizing XP. This practice first time proposed as a long research paper in APSO workshop at ICSE 2008 conference and yet it is the only proposed and applicable method for customizing XP. Although it is specifically a solution for XP, this practice has the capability of extending to other methodologies. At first glance, this practice seems to be in the category of static method adaptation but experiences with RDP Practice says that it can be treated like dynamic method adaptation. The distinction between static method adaptation and dynamic method adaptation is subtle.[13] The key assumption behind static method adaptation is that the project context is given at the start of a project and remains fixed during project execution. The result is a static definition of the project context. Given such a definition, route maps can be used in order to determine which structured method fragments should be used for that particular project, based on predefined sets of criteria. Dynamic method adaptation, in contrast, assumes that projects are situated in an emergent context. An emergent context implies that a project has to deal with emergent factors that affect relevant conditions but are not predictable. This also means that a project context is not fixed, but changing during project execution. In such a case prescriptive route maps are not appropriate. The practical implication of dynamic method adaptation is that project managers often have to modify structured fragments or even innovate new fragments, during the execution of a project (Aydin et al., 2005).[13]

Measuring agility

While agility can be seen as a means to an end, a number of approaches have been proposed to quantify agility. Agility Index Measurements (AIM)[14] score projects against a number of agility factors to achieve a total. The similarly-named Agility Measurement Index,[15] scores developments against five dimensions of a software project (duration, risk, novelty, effort, and interaction). Other techniques are based on measurable goals.[16] Another study using fuzzy mathematics[17] has suggested that project velocity can be used as a metric of agility. There are agile self assessments to determine whether a team is using agile practices (Nokia test,[18] Karlskrona test,[19] 42 points test[20]).

While such approaches have been proposed to measure agility, the practical application of such metrics has yet to be seen.

Experience and reception

One of the early studies reporting gains in quality, productivity, and business satisfaction by using Agile methods was a survey conducted by Shine Technologies from November 2002 to January 2003.[21] A similar survey conducted in 2006 by Scott Ambler, the Practice Leader for Agile Development with IBM Rational's Methods Group reported similar benefits.[22] In a survey conducted by VersionOne in 2008, 55% of respondents answered that Agile methods had been successful in 90-100% of cases.[23] Others claim that agile development methods are still too young to require extensive academic proof of their success.[24]

Suitability

Large-scale agile software development remains an active research area.[25][26]

Agile development has been widely documented (see Experience Reports, below, as well as Beck[27] pg. 157, and Boehm and Turner[28]) as working well for small (<10 developers) co-located teams.

Some things that can negatively impact the success of an agile project are:

Several successful large-scale agile projects have been documented. BT has had several hundred developers situated in the UK, Ireland and India working collaboratively on projects and using Agile methods.

Barry Boehm and Richard Turner suggest that risk analysis be used to choose between adaptive ("agile") and predictive ("plan-driven") methods.[28] The authors suggest that each side of the continuum has its own home ground as follows:

Agile home ground:[28]

Plan-driven home ground:[28]

Formal methods:

Experience reports

Agile development has been the subject of several conferences. Some of these conferences have had academic backing and included peer-reviewed papers, including a peer-reviewed experience report track. The experience reports share industry experiences with agile software development.

As of 2006, experience reports have been or will be presented at the following conferences:

[36] )

See also

References

  1. ↑ 1.0 1.1 1.2 Beck, Kent; et al. (2001). "Manifesto for Agile Software Development". Agile Alliance. http://agilemanifesto.org/. Retrieved 2010-06-14. 
  2. ↑ 2.0 2.1 Gerald M. Weinberg, as quoted in Larman, Craig; Basili, Victor R. (June 2003). "Iterative and Incremental Development: A Brief History". Computer 36 (6): 47–56. doi:10.1109/MC.2003.1204375. ISSN 0018-9162. "We were doing incremental development as early as 1957, in Los Angeles, under the direction of Bernie Dimsdale [at IBM's ServiceBureau Corporation]. He was a colleague of John von Neumann, so perhaps he learned it there, or assumed it as totally natural. I do remember Herb Jacobs (primarily, though we all participated) developing a large simulation for Motorola, where the technique used was, as far as I can tell .... All of us, as far as I can remember, thought waterfalling of a huge project was rather stupid, or at least ignorant of the realities. I think what the waterfall description did for us was make us realize that we were doing something else, something unnamed except for 'software development.'". 
  3. ↑ Edmonds, E. A. (1974). "A Process for the Development of Software for Non-Technical Users as an Adaptive System". General Systems 19: 215–18. 
  4. ↑ Larman, Craig (2004). Agile and Iterative Development: A Manager's Guide. Addison-Wesley. p. 27. ISBN 9780131111554 
  5. ↑ Kent Beck, Mike Beedle, Arie van Bennekum, Alistair Cockburn, Ward Cunningham, Martin Fowler, James Grenning, Jim Highsmith, Andrew Hunt, Ron Jeffries, Jon Kern, Brian Marick, Robert C. Martin, Steve Mellor, Ken Schwaber, Jeff Sutherland, and Dave Thomas
  6. ↑ Beck, Kent; et al. (2001). "Principles behind the Agile Manifesto". Agile Alliance. http://www.agilemanifesto.org/principles.html. Retrieved 2010-06-06. 
  7. ↑ Anderson, David (2005). "Declaration of Interdependence". http://pmdoi.org. 
  8. ↑ Beck, Kent (1999). "Embracing Change with Extreme Programming". Computer 32 (10): 70–77. doi:10.1109/2.796139. 
  9. ↑ Boehm, B.; R. Turner (2004). Balancing Agility and Discipline: A Guide for the Perplexed. Boston, MA: Addison-Wesley. ISBN 0-321-18612-5.  Appendix A, pages 165-194
  10. ↑ 10.0 10.1 Aydin, M.N., Harmsen, F., Slooten, K. v., & Stagwee, R. A. (2004). An Agile Information Systems Development Method in use. Turk J Elec Engin, 12(2), 127-138
  11. ↑ Abrahamsson, P., Warsta, J., Siponen, M.T., & Ronkainen, J. (2003). New Directions on Agile Methods: A Comparative Analysis. Proceedings of ICSE'03, 244-254
  12. ↑ Abrahamsson, P., Salo, O., Ronkainen, J., & Warsta, J. (2002). Agile Software Development Methods: Review and Analysis. VTT Publications 478
  13. ↑ 13.0 13.1 Aydin, M.N., Harmsen, F., Slooten van K., & Stegwee, R.A. (2005). On the Adaptation of An Agile Information(Suren) Systems Development Method. Journal of Database Management Special issue on Agile Analysis, Design, and Implementation, 16(4), 20-24
  14. ↑ "David Bock's Weblog : Weblog". Jroller.com. http://jroller.com/page/bokmann?entry=improving_your_processes_aim_high. Retrieved 2010-04-02. 
  15. ↑ "Agility measurement index". Doi.acm.org. http://doi.acm.org/10.1145/1185448.1185509. Retrieved 2010-04-02. 
  16. ↑ Peter Lappo; Henry C.T. Andrew. "Assessing Agility". http://www.smr.co.uk/presentations/measure.pdf. Retrieved 2010-06-06. 
  17. ↑ Kurian, Tisni (2006). "Agility Metrics: A Quantitative Fuzzy Based Approach for Measuring Agility of a Software Process" ISAM-Proceedings of International Conference on Agile Manufacturing'06(ICAM-2006), Norfolk, U.S.
  18. ↑ Joe Little (2007-12-02). "Nokia test, A Scrum specific test". Agileconsortium.blogspot.com. http://agileconsortium.blogspot.com/2007/12/nokia-test.html. Retrieved 2010-06-06. 
  19. ↑ Mark Seuffert, Piratson Technologies, Sweden. "Karlskrona test, A generic agile adoption test". Piratson.se. http://www.piratson.se/archive/Agile_Karlskrona_Test.html. Retrieved 2010-06-06. 
  20. ↑ "How agile are you, A Scrum specific test". Agile-software-development.com. http://www.agile-software-development.com/2008/01/how-agile-are-you-take-this-42-point.html. Retrieved 2010-06-06. 
  21. ↑ "Agile Methodologies Survey Results" (PDF). Shine Technologies. 2003. http://www.shinetech.com/attachments/104_ShineTechAgileSurvey2003-01-17.pdf. Retrieved 2010-06-03. "95% [stated] that there was either no effect or a cost reduction . . . 93% stated that productivity was better or significantly better . . . 88% stated that quality was better or significantly better . . . 83% stated that business satisfaction was better or significantly better" 
  22. ↑ Ambler, Scott (August 3, 2006). [httphttp://www.drdobbs.com/architecture-and-design/191800169;jsessionid=2QJ23QRYM3H4PQE1GHPCKH4ATMY32JVN?queryText=agile+survey "Survey Says: Agile Works in Practice"]. Dr. Dobb's. httphttp://www.drdobbs.com/architecture-and-design/191800169;jsessionid=2QJ23QRYM3H4PQE1GHPCKH4ATMY32JVN?queryText=agile+survey. Retrieved 2010-06-03. "Only 6 percent indicated that their productivity was lowered . . . No change in productivity was reported by 34 percent of respondents and 60 percent reported increased productivity. . . . 66 percent [responded] that the quality is higher. . . . 58 percent of organizations report improved satisfaction, whereas only 3 percent report reduced satisfaction." 
  23. ↑ "The State of Agile Development" (PDF). VersionOne, Inc.. 2008. http://www.versionone.com/pdf/3rdAnnualStateOfAgile_FullDataReport.pdf. Retrieved 2010-07-03. "Agile delivers" 
  24. ↑ "Answering the "Where is the Proof That Agile Methods Work" Question". Agilemodeling.com. 2007-01-19. http://www.agilemodeling.com/essays/proof.htm. Retrieved 2010-04-02. 
  25. ↑ Agile Processes Workshop II Managing Multiple Concurrent Agile Projects. Washington: OOPSLA 2002
  26. ↑ "Supersize Me" in Dr. Dobb's Journal, February 15, 2006.
  27. ↑ Beck, K. (1999). Extreme Programming Explained: Embrace Change. Boston, MA: Addison-Wesley. ISBN 0-321-27865-8. 
  28. ↑ 28.0 28.1 28.2 28.3 Boehm, B.; R. Turner (2004). Balancing Agility and Discipline: A Guide for the Perplexed. Boston, MA: Addison-Wesley. pp. 55–57. ISBN 0-321-18612-5. 
  29. ↑ "Supersize Me". Sdmagazine.com. http://www.sdmagazine.com/documents/s=10020/sdm0603g/0603g.html. Retrieved 2010-06-06. 
  30. ↑ Schaaf, R.J. (2007). "Agility XL", Systems and Software Technology Conference 2007, Tampa, FL
  31. ↑ "Bridging the Distance". Sdmagazine.com. http://www.sdmagazine.com/documents/s=7556/sdm0209i/sdm0209i.htm. Retrieved 2010-06-06. 
  32. ↑ Martin Fowler. "Using an Agile Software Process with Offshore Development". Martinfowler.com. http://www.martinfowler.com/articles/agileOffshore.html. Retrieved 2010-06-06. 
  33. ↑ [The Art of Agile Development James Shore & Shane Warden pg 47]
  34. ↑ 2000
  35. ↑ "2006". Virtual.vtt.fi. http://virtual.vtt.fi/virtual/xp2006/. Retrieved 2010-06-06. 
  36. ↑ "2010". Xp2010.org. http://www.xp2010.org/. Retrieved 2010-06-06. 
  37. ↑ 2001
  38. ↑ 2002
  39. ↑ 2003
  40. ↑ 2004
  41. ↑ "Agile Development Conference". Agile200x.org. http://www.agile200x.org/. Retrieved 2010-06-06. 

Further reading

External links

Software Engineering
Fields
Requirements analysis • Systems analysis • Software design • Computer programming • Formal methods • Software testing • Software deployment • Software maintenance
Concepts
Data modeling • Enterprise architecture • Functional specification • Modeling language • Programming paradigm • Software • Software architecture • Software development methodology • Software development process • Software quality • Software quality assurance • Software archaeology • Structured analysis
Orientations
Agile • Aspect-oriented • Object orientation • Ontology • Service orientation • SDLC
Models
Development models
Agile • Iterative model • RUP • Scrum • Spiral model • Waterfall model • XP • V-Model • Incremental model • Prototype model
Other models
Automotive SPICE • CMMI • Data model • Function model • Information model • Metamodeling • Object model • Systems model • View model
Modeling languages
IDEF • UML
Software
engineers
Kent Beck • Grady Booch • Fred Brooks • Barry Boehm • Ward Cunningham • Ole-Johan Dahl • Tom DeMarco • Martin Fowler • C. A. R. Hoare • Watts Humphrey • Michael A. Jackson • Ivar Jacobson • Craig Larman • James Martin • Bertrand Meyer • David Parnas • Winston W. Royce • Colette Rolland • James Rumbaugh • Niklaus Wirth • Edward Yourdon • Victor Basili
Related fields
Computer science • Computer engineering • Enterprise engineering • History • Management • Mathematics • Project management • Quality management • Software ergonomics • Systems engineering